The rapid expansion of information service and data processing industries has resulted in a need for computer systems to manage and store large amounts of data. As an example, financial service industry businesses such as banks, mutual fund companies or the like often operate large and complex data processing systems that require access to many hundreds of gigabytes or even terabytes of data. Data storage system developers have responded to these types of data storage requirements by integrating large capacity data storage systems, data communications devices and computer systems into networks called “storage networks” or “storage area networks” (SANs). A storage area network is a collection of data storage systems that are networked with a number of host computer systems that operate as servers to access data stored in the data storage systems.
Elements of a typical conventional storage area network implementation include one or more connectivity devices such as high speed data switches or routers that interconnect the various data storage systems to each other and to one or more host or server computer systems (servers) that require access to (e.g., read and/or write) the data in the data storage systems on behalf of client software applications and/or client computer systems.
A developer or administrator of such a storage area network environment may install one or more distributed storage area network management software applications within the storage area network to manage or administer the various elements (i.e., devices, computer systems, storage systems, etc.) that operate within the storage area network. A network manager (i.e., a person) responsible for management of the storage area network operates the network management software application to perform management tasks such as performance monitoring, network analysis and remote configuration and administration of the various components operating within the storage area network.
A typical conventional storage area network management software application may have several different software components that execute independently of each other on different computer systems but that collectively interoperate together to perform network management. As an example, conventional designs of storage area network management applications can include console, server, agent and storage software components.
Generally, the server component operates as a central control process within the storage area network management application and coordinates communication between the console, storage (i.e. store) and agent components. The console component often executes within a dedicated storage area network management workstation to allow the network administrator to visualize and remotely control and manage the various elements within the storage area network that are graphically represented within the console. Agent components execute on host computer systems such as servers within the storage area network to manage storage area network elements. As an example, there may be different respective agents specifically designed (e.g., coded) to remotely manage and control data storage systems, database applications, switches, and so forth. Agent components receive remote management commands from the server component and apply functionality associated with those management commands to the managed elements within the storage area network for which those agents are designated to manage. Agents are also responsible for periodically collecting configuration or management data concerning the storage area network elements that those agents are responsible for management. Agents can transmit this collected management data back to a storage component. The storage component receives the collected management data from the agents and processes and stores this information into a storage area network management database. The console component can obtain current information such as performance, capacity, load or other data concerning managed elements within the storage area network by accessing the element configuration data in the network management database.
Different components of the storage area network may be located remotely from each other. For example, the agents may be located remotely from the server and storage components, and communicate with them over a network such as the Internet. A firewall may be used on the network between the agent and other parts of the system. The firewall is used to provide a measure of security such that the server and storage components are not available (such as by having their IP addresses published) to other people or systems. NAT is now commonplace and comes from a number of factors including a shortage of IP addresses, security needs and ease and flexibility of network administration.
Enterprise Storage Networks are large and complex environments that include various elements such as storage arrays, switches, hosts and databases all inter-networked. These elements occur in several hundreds in such environments. These elements in turn may consist of several hundred thousands of manageable elements such as storage devices, storage and switch ports, database instances, host devices and file systems, and the like. Management of such environments is a daunting task and typically requires Storage Resource Management solutions such as EMC ControlCenter® (ECC) storage area network management family of products, available from EMC Corporation of Hopkinton, Mass. ECC includes agents that are deployed for the purpose of gathering data about various storage elements, components that process and persist data, applications that use persisted information to enable the management of these environments. ECC identifies the entire process of retrieving the collected data from agents to data persistence as a transaction. The sizes of collected data range from a few kilobytes to several hundred megabytes. Processing these from as many agents as there are deployed can be complex, time-consuming and failure-prone.